Skip to content

feat: open Insta360 .insp photos as spheres - #47

Open
dynobo wants to merge 10 commits into
mainfrom
feat/insta360-insp
Open

dynobo wants to merge 10 commits into
mainfrom
feat/insta360-insp

Conversation

@dynobo

@dynobo dynobo commented Nov 8, 2026 •

Copy link
Copy Markdown
Owner

Implements the viewer half of #45: .insp photos open as spheres, straight from the file.

An .insp is a JPEG holding both lenses' fisheye circles side by side, plus a trailer at the end that says where each circle sits, how wide each lens sees and how the two lenses are rotated against each other. This reads that trailer in Rust and hands the numbers to the viewer, which projects the two circles on the GPU while drawing. There is no stitched intermediate image and nothing is written next to the original, so nothing is resampled twice and the full resolution of each circle stays available at whatever the viewport can show.

What changes for the user

  • Opening an Insta360 photo shows the sphere instead of two circles.
  • A new Projection section in the main menu, and Ctrl+D, switch between Equirectangular and Dual Fisheye. Both are always offered, for every image.
  • Without a calibration (any other dual fisheye image, or an .insp whose trailer was stripped) the viewer assumes the standard layout, two 180° circles filling the frame, and says so in a notification.
  • .insp gets a file type of its own (image/x-insta360-insp) and SphereView declares that it handles it, so the desktop can offer it for these files instead of handing them to a JPEG viewer.

How it is built

src/insta360.rs Trailer reader. Records are walked backwards from the end; the padding between the last record and the footer is not announced, so every plausible width is tried and the walk that lands exactly on the trailer start wins. The metadata record is parsed as protobuf without a schema, and the calibration as plain text.
src/image.rs Recognises the format by content, not by name, so a renamed file still works. Hands the viewer the JPEG cut off before the trailer.
resources/photosphereviewer/src/viewer.js An adapter extending PSV's equirectangular one. Only images that ask for a projection take the dual fisheye path; ordinary panoramas, XMP GPano data and partial panoramas still go through PSV untouched.

The shader is a port of PSV's own MIT dual-fisheye adapter, with its hardcoded 180° and circle size replaced by the per-file calibration. Levelling is not baked into pixels — when the camera's motion sensor reading is used, it will go through sphereCorrection, which is an exact rotation.

Licence

Implemented from published measurements — golblick's format notes, insta360-stitch (MIT OR Apache-2.0) and exiftool — rather than ported from golblick's AGPL code, so this stays MIT. The one dependency added is three, which PSV already requires.

Testing

cargo fmt --all -- --check
cargo test                     29 tests
cargo clippy --all-targets     clean in the new code
  • tests/insta360_trailer.rs: the record walk, the padding ambiguity (a 32-byte pad also reads as five zero-length records, which a naive walk trips over), truncation, a bad magic, a trailer size running past EOF, and a calibration string taken verbatim from a real Insta360 ONE file.
  • tests/fixtures/synthetic.insp: generated by tools/make_synthetic_insp.py, so it is ours and needs no camera. It goes through the real loader in src/image.rs.
  • SPHEREVIEW_TEST_INSP=/path/to/photo.insp cargo test --test insta360_trailer runs against a real camera file locally. None is committed: .insp originals are photographs, and no vendor grants redistribution.
  • tests/mime_registration.rs cross-checks the MIME database against the desktop entry for every extension the application opens, so a new format cannot be added to one and forgotten in the other. I also built a throwaway MIME database from the new XML and confirmed GLib resolves synthetic.insp to image/x-insta360-insp while a plain .jpg stays image/jpeg, and that desktop-file-validate accepts the entry.

Beyond the tests I ran the app under Xvfb with software GL and screenshotted it: FRONT reads correctly at the default view, RIGHT at yaw 90 with the seam running through the centre of the view without doubling, BACK at yaw 180, and at pitch 88 the +30° parallel is a clean circle with meridians radiating from the zenith. That check is what caught two bugs in the shader: a missing varying, and a mirrored x axis, because PSV builds its sphere with scale(-1,1,1) and compensates inside its own shaders.

Known limits

  • Two conventions are still assumptions, both declared as uniforms so they are one-line changes: which way each lens's image runs along x, and that lens 0 faces +Z. The synthetic fixture cannot settle them because it is generated with the same mapping. A real OneR/X3/X5 file would.
  • No seam routing for parallax: near-field objects that sit in both lenses can double at the seam. golblick solves this with a search; this does not attempt it.
  • No level sphereCorrection yet — the motion sensor reading is parsed but unused.
  • Double-click needs the MIME database installed, not just committed: an .insp resolves to image/jpeg by content sniffing, so the new type only wins once io.github.dynobo.sphereview.mime.xml is installed and update-mime-database has run. The Flathub manifest pins a release tag and does not install the XML yet, so that is a second pull request against a release containing this one. Magic matching is not an option: shared-mime-info offsets count from the start of the file and the Insta360 trailer is at the end, so the type is matched on *.insp.
  • .insv video is out of scope.

Refs #45

dynobo added 10 commits November 9, 2026 00:21
An .insp is a JPEG holding both lenses' fisheye circles side by side, with a
trailer at the end saying where each circle sits and how wide each lens sees.
Reading that trailer is enough to project the pair onto the sphere while it is
drawn, so the file's own pixels are what you see: no baked panorama, no extra
resampling, nothing written to disk.

- src/insta360.rs reads the trailer: records walked backwards, unknown padding
  width, protobuf without a schema, and the lens calibration as plain text.
- src/image.rs recognises the format by content rather than by name, cuts the
  JPEG off before the trailer, and passes the calibration to the viewer.
- viewer.js adds an adapter that extends PSV's equirectangular one, so ordinary
  panoramas, XMP data and partial panoramas still go through PSV untouched.
- The projection is offered for every image, under a new "Projection" menu
  section and Ctrl+D. Without a calibration the viewer assumes the standard
  layout of two 180 degree circles and says so in a notification.

The format is implemented from published measurements (golblick's format notes,
insta360-stitch, exiftool) rather than ported from AGPL code, so the licence
stays MIT.

Tests cover the trailer walk, the padding ambiguity, truncation, a calibration
string taken from a real camera file, and a generated .insp checked through the
image loader end to end. A real-file test runs when SPHEREVIEW_TEST_INSP is set;
no camera file is committed, because .insp originals are photographs whose
copyright is not ours to give away.
An .insp resolves to image/jpeg by content sniffing, so double-clicking one
opens whatever handles JPEGs - a viewer that shows the two fisheye circles.
Give the format a type of its own and say SphereView handles it.

The shared-mime-info magic rules can only match at offsets from the start of a
file, and the marker that identifies an .insp (the camera's trailer) is at the
end, so the type is matched on the extension. Deliberately not declared as a
subclass of image/jpeg: readers of the parent type show the raw circles, and
they should not be offered for these files.

Nothing changes for users until a build installs both files. The Flathub
manifest has to install the XML and refresh the MIME cache, which is a separate
change against the release that first ships this.

tests/mime_registration.rs keeps the two friends honest: every extension the
application opens has to appear in the desktop entry's MimeType, and the
Insta360 type additionally in the MIME file.
The fisheye shader took three conventions from the file's calibration
that it had wrong; each was found by measuring two real photos (a ONE RS
and a ONE X2) against their ~15 degree lens overlap, not by reading the
format, which documents none of this.

- The z rotation in the calibration is the direction a lens faces - the
  two lenses differ by about 180 degrees - and not the roll of its image.
  Using it as a roll turned one hemisphere 180 degrees against the other:
  the reported "one side upside down, front is back".
- `fov_deg` is what a lens sees across its circle, so the rim is at half
  that angle from the axis. Treating 194 degrees as a half-angle
  stretched the sphere by 2x radially.
- The x axis sign keeps the photo unmirrored. Settled against a
  parking-garage sign visible in the photo, because combining PSV's
  frame, the mirrored sphere geometry and three's camera handedness is
  easy to get wrong.

The adapter is now split by concern: `adapter.js` only dispatches between
the two projections, `dual-fisheye.js` holds the shader and its uniforms,
`viewer.js` is the bootstrap. `theta_max_deg` is `fov_deg` end to end with
the docs corrected, the deprecated core DualFisheyeAdapter was never the
right parent, and the synthetic fixture is regenerated because it had
been built with the same wrong FOV assumption - which is why it agreed
with the bug.

`npm test` pins the three conventions and now runs as part of the build.

Known and unrelated: which way is up in the world is the camera's
attitude when the photo was taken (this ONE RS is stored upside down, the
ONE X2 rolled 90 degrees). It is in neither GPano nor the calibration.
Insta360's app levels every photo, so the question comes up as soon as the
spheres render correctly. For the ONE RS and ONE X2 measured here the
attitude is not in the file: no gyro record, no XMP GPano pose, an empty
EXIF MakerNote, and metadata rotations of a degree or two that are lens
alignment. Both files were shot rolled, by 180 and by 90 degrees, which
also rules out a per-model convention.
Insta360 photos carry no record of which way was up when they were taken,
and the cameras have no up of their own, so a sphere arrives rolled by
whatever angle the camera was held at: the ONE RS photo measured is
upside down, the ONE X2 one is a quarter turn over. Levelling therefore
needs a control, not a parser.

Turns the content a quarter turn per click through sphereCorrection.roll,
which rotates about the forward axis - the axis the camera rolled about -
and works for equirectangular panoramas too. The roll stays in the
viewer's config, so it survives loading another file.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant